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ABSTRACT 

This document is part of a five-module training package to 
help employment and training service providers comply with the Workforce 
Investment Act (WIA) of 1998 and develop a one-stop training and employment 
services system. It contains the participant workbook, trainer manual, and 
activity worksheets for a module on collecting and using customer feedback. 
The training manual presents a detailed lesson plan that includes 
step-by-step instructions for using the following tools and techniques to 
deliver the module: presenting overhead slides, having participants complete 
exercises; writing information on flip charts; using small group discussions; 
and using full group discussion. The workbook activities and worksheets are 
designed to help participants understand why customer feedback should be the 
foundation for designing a responsive service system and to teach 
participants to collect and analyze customer feedback. Participants are 
presented with a four-phase customer feedback collection consisting of the 
following actions: (1) prepare to collect feedback by identifying services 
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and/or products, customer values, customer requirements/attributes, and 
product features; (2) use qualitative and quantitative techniques to collect 
proactive and reactive feedback; (3) use qualitative and qualitative methods 
to analyze feedback; and (4) utilize customer feedback in policy and 
procedural decisions, to make individual daily improvements, and as part of a 
continuous improvement system. (MN) 
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Partnering for Quality under WIA 

Module 3. Customer Feedback 



Trainer Manual Guidelines 

This manual provides a step-by-step approach to training on 
Collecting and Using Customer Feedback. This session is the third 
in a series of five training modules designed to help local workforce 
development agencies collaborate to improve customer services. 
The complete curriculum on Partnering for Quality under WIA 
includes: 

• Module 1 . Reviewing System-Building Progress and 
Developing a Blueprint for WIA Transition 

• Module 2. Partnering and Organizational Change 

• Module 3. Collecting and Using Customer Feedback 

• Module 4. Designing a System for the Delivery of Integrated 
Services 

• Module 5. Building a Process for Continuous Improvement 

Each module has a series of companion products. In addition to 
this trainer manual, products developed for Module 3 on Collecting 
and Using Customer Feedback include overhead transparencies and 
a student workbook. 

The best place to start with this guide is to read it through once. 

The guide is designed to be easy to understand. Following are 
points for reading and using the guide. 

Sentences typed in bold italics are intended to be said by 
the trainer to the group . 

• Sentences with a bullet in front of them are action 
items that the trainer needs to do . 
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Icons on the left hand side of the guide provide a visual view of the 
type of activity that is taking place. The Icon Key is provided 
below. 




• Use overhead slide. 




• Have participants complete exercise. 


H 


• Write information on flip chart. 


MiSk 


• Use small group work/discussion. 




• Use full group discussion. 
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Write “Welcome” and your name and phone number bn a flip 
chart at front of room prior to attendees arriving. Also include 
your e-mail address if you have one. 



WELCOME! I’m glad you could be with us today. 



• Introduce yourself and give a brief description of your 
background and qualifications to lead this training module. 




Put on title overhead for Module 3. 



SPR SOCIAL POLICY RESEARCH ASSOCIATES 

Partnering to Improve 
Customer Services: 

A Tool Kit for Workforce 
Investment Partners 

Module 3 

COLLECTING AND USING CUSTOMER 
FEEDBACK 



This session mil cover the material in your Participant 
Workbook under Module 3. Collecting and Using Customer 
Feedback. 




Put on Overhead 3-2. 
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Customer/Provider Activity 

Purpose 

• To learn the importance of using 
customer feedback to develop a 
product. 



3-2 




The purpose of the beginning activity is to learn the importance 
of using customer feedback to develop a product . 




Put on Overhead 3.3. 



• Display one bullet point at a time. 



Customer/Provider Activity 

Instructions: 

• Find a partner-Designate one person as "A” 
and one person as "B.” "A" s play the role of 
the customer; "B"s play the role of the 
provider. 

• The trainer will hand out picture to “A"s. 

• "A"s will review a picture and develop a one- 
word description of what they want M B"s to 
draw for them. 

• "B"s have 2 minutes to draw the picture 
based on the one-word description. 



• Display the first point on the overhead, and say: 

Find a partner . Decide which of you will be partner “A 99 and 
which of you will be partner “B. 99 
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• After you have given them time to do that, ask that the “A’s” 
raise their hands. Then have the “B’s” raise their hands so that 
you can determine that everyone has a role. If there is one 
person without a partner, have them join another group and 
designate 1 “A” and 2 “B’s.” 

• Display the second point on the overhead, and say: 



The “A ’s” will play the role of the customer and the “B’s” will 
play the role of the provider. 



• State that all of the “A’s” (customers) need to come to the front 
to receive a picture. Distribute blank sheets of paper to the 
“B’s” (providers). Once the customers have the picture, have 
them jointly — but without the “B’s” hearing them — determine 

a one- word description for the picture. Once they have come to 
consensus on that one-word description, have them return to 
their seats while keeping the picture secret from the provider. 

• Display the third point on the overhead, and say to the “B’s” 
(providers): 



The customers are going to ask you to draw a picture. The 
objective is to make a drawing that looks like the picture they are 
holding. However, they are only going to give you a one-word 
description of the picture. Do your best. The point of this 
exercise is not to see who is the best artist, but to get the best 
likeness (key features) of the picture that is held by the customer. 



• Display the fourth point on the overhead. Tell the providers 
that they have two minutes to draw the picture. Set the timer 
for two minutes. Make sure no other communication happens 
between the “A’s” and the “B’s” while the “B’s” are drawing. 
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Put on Overhead 



3.4. 



• Display one bullet point at a time. 



Customer/Provider Activity 

Instructions, Continued 

• "A"s review drawing and determine if it meets 
their criteria. 

• 'TV's provide additional verbal feedback to 
"B"s and use the next 2 minutes to help them 
to create the picture they require. 

• Debrief the process. 

3-4 



• Display the first point on the overhead and tell “A’s” to review 
the picture drawn by the “B’s.” Have the “A’s” determine if it 
is exactly the same as their secret picture. 

• If not, display the second point on the overhead. Instruct the 
“A’s” to provide feedback and all the necessary information to 
the “B’s” to insure that the picture drawn by the “B’s” is 
exactly like the secret picture. Set the timer for 2 minutes. 



• Display the last point on the overhead and help the participants 
debrief the process. 





Facilitate group discussion on the activity they just completed. 



Put on Overhead 3-5. 
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Customer/Provider Activity 

Review Questions 


• "B"s - When you were 


"A"s - Did you get what you 


unable to check with your 


expected with the first 


customer, did you provide 


picture? 


the product they were 


"A"s - What difference did 


expecting? 


you find when you were abie 


• "B"s - How did you feel/what 


to communicate with the 


did you think without 


provider as they were 


information from your 


drawing the picture? 


customer? Was it hard or 


As a customer (A), is your 


easy to know what to draw? 


input important to get what 


• "B"s - How did you feel/what 


you want and need? As a 


did you think when you 


Provider (B), do you need 


worked with your customer? 


the customer’s input to know 


What was easy? What was 


what the product should look 


hard? 


like? 3-5 



• Ask the questions on the overhead to the appropriate parties. 
Use the feedback from the “A’s” and “B’s” to determine how 
they felt doing the activity and what they learned. 



Drawing pictures is not as complicated as developing a One-Stop 
center or system. The activity you have just completed makes it 
clear that in developing a One-Stop system it is even more 
important to have as much customer input as possible , so the 
“picture” comes out looking like what the customer has in 
his/her mind. 



• Put on Overhead 3-6.. 



Social Policy Research Associates 



12 



Partnering for Quality under WIA 



Trainer Manual 



Module 3. Customer Feedback 



The Customer’s Perspective 

When one has no stake in the way things are, 
when one's needs or opinion are provided no 
forum, when one sees oneself as the object 
of unilateral actions, it takes no particular 
wisdom 

to suggest that one would rather be 
elsewhere. 

Seymour Sarason, 1990 

3-6 




Read the quotation and discuss how this quotation relates to 
developing and managing a One-Stop center/system. 



• Discuss how internal and external customers of the One-Stop 
need to provide input or they will not utilize the system. 




• Put on Overhead 3.7. 
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• Discuss how concentrating on customer requirements in 

developing the One-Stop will lead to products and services that 
lose the identity of each of the partner agencies and take on a 
new One-Stop identity. 



• Discuss how to reduce agency “turfism” by concentrating on 
developing the products of the One-Stop to meet customer 
requirements. 




Introduce the learning objectives by saying: 



In this session , we will review how to use customer feedback to 
create the best possible One-Stop for our customers . 



Please introduce yourself and describe one thing that you would 
like to learn in this session . 




Record the responses on a flip chart and hang the completed 
pages around the room to return to throughout the session. 



Display Overheads 3-8, 3-9, and 3-10.. 



Learning Objectives 

• Learn why customer feedback is a 
foundation for One-Stop services. 

• Understand the process of customer 
feedback. 

• Understand the 4 phases of customer 
feedback. 

• Learn how to identify the product(s), 
customer(s), requirement(s), and standard(s) 
of the One-Stop system. 

3-8 
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Learning Objectives, Continued 

• Understand the two types of customer 
feedback and the methods to collect each 
type. 

• Learn the methods to analyze customer 
feedback. 

• Explore how customer requirements impact 
program design, customer flow, staffing 
patterns, office layout. 

3-9 



Learning Objectives, Continued 



• Learn how to create a team action plan to 
begin designing/refining your One Stop's 
Customer Feedback Process. 



3-10 



• Review the learning objectives on the overheads one by one. 
Compare them to the stated requests of the participants. 




Put on Overhead 3-11. 
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Agenda 

• Introductions/Welcome 

• Why Use Customer Feedback to Design One- 
Stop Centers/Systems? 

• The Cycle of Customer Feedback 

: 4 Phases to the Customer Feedback Cycle 

- Prepare to Collect 

- Collect 

- Analyze 

- Utilize 

• Setting Goals for Improving Customer 

Services 3-11 




• Relate the agenda items to the stated requests of the 

participants and where we currently are on the agenda. Note 
that the next step on the agenda is the “cycle of customer 
feedback.” 



What do most organizations do when they are told that they must 
pay attention to “ customer satisfaction? 

• Once an attendee states “do a survey,” put on Overhead 3-12 
and ask the attendees to describe where on the listed items they 
find something that describes “do a survey.” 



The Customer Feedback 
Cycle 



ID Services/Products 
ID Customers of the 
Services/Products 
ID What Customer Values 
About Each 
Service/Product 
ID Customer Requirements 
(Attributes) for Each 
ID Product Features 
ID Customer Standards for 
the Requirements 
ID What to Measure 



ID What Data to Collect 

Qualitative? 

Quantitative? 

ID How to Measure - Which 
Statistical Process Control 
Tools 

—To Collect 
— To Analyze 

Collect the Data 
Analyze the Data 

— Meet/exceed Customer 
Standard? 

- Not Meet Customer Standard? 

Make Improvements 
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• Once they say “collect the data,” ask them to count how many 
steps preceded that step. 

If an organization starts at (( collect the data 99 they have missed 9 
important steps . 

• Review each of the steps one by one. Describe how each step 
is an important part of the cycle. 

The steps in the customer feedback cycle can be divided into 4 
Phases . 



4 Phases to the 
Customer Feedback Cycle 

• Prepare to Collect 

• Collect 

• Analyze 

• Utilize 



• Describe each phase. 

Find a partner and discuss which of the steps on Overhead 3-12 
relate to each of the phases on Overhead 3-13 . 

• Have one of the partner groups report out which steps go with 
which phase. 

We will be reviewing each of the phases individually. 




• Put on Overhead 3-13. 



3-13 
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• Put on Overheads 3-14 and 3-15. 



Customer Feedback 
Phase 1: Prepare to Collect 

FOCUSING THE PROCESS - ("THE MISSING 
PIECE") 

• Identify services and/or products. 

• Identify customers of each. 

• Identify what customer values about each. 

• Identify customer requirement(s)/attribute(s) 
for each. 

• Identify product features. 

3-14 



Customer Feedback 
Phase 1 : Prepare to Collect 

Continued 

• Identify standards for each requirement. 

• Identify what to measure - what do you want 
to know? 

• Identify what data to collect. 

-- Qualitative? 

— Quantitative? 

• Identify how to measure - which statistical 
process control tools to use to collect. 

3-15 



This phase — “prepare to collect” — is the most forgotten and 
neglected phase. Many people tend to leave it out altogether. 
However, this phase is what ensures that the right data is 
collected in the proper manner. 

• State that if this phase is neglected, organizations may not be 
able to analyze or utilize the data once it is collected. This 
renders the entire customer feedback process useless. 

• Discuss using the following examples. 
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Suppose an organization has a 95% customer satisfaction rate . 
What should the organization do to satisfy the other 5% of its 
customers? 




• Without having specific information about how well the 
organization is doing on its product requirements, the 
organization does not have the necessary information to make 
improvements that will increase customer satisfaction. 

• Going through the steps in the "prepare to collect” phase will 
provide the necessary information to ensure that the right data 
is being collected 

• Put on Overhead 3-16. 



Customer Feedback 
Phase 1 : Prepare to Collect 

Continued 

ProducfCriteria 

• It must be a noun. 

• It must be made plural with an “s” or 
“es.” 

• It must be deliverable (you must be able 
to give it to someone). 

• It must be yours to give. 



The first step in preparing to collect customer feedback is to 
determine the products that the One-Stop provides . 



• Review the four criterion suggested by Ken Miller to determine 
if something is a product: 




Put on Overhead 3-17. 
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Customer Feedback 
Phase 1 : Prepare to Collect 

Continued 

• Activity Worksheet 3-A -Identifying 
Customer Standards - will be used for 
an exercise that will ask you to : 

- Identify products. 

- Identify customers. 

- Identify product outcomes and features. 

- Identify customer requirements. 

- Define customer standards. 

3-17 



• Help participants find Activity Worksheet 3-A, Identifying 
Customer Standards . 




• Put on Overhead 3-18. 



Customer Feedback 
Phase 1 : Prepare to Collect 

Continued 

Identify Products 

• Working with your table, and using the 
criteria, identify products that your One- 
Stop: 

-delivers currently. 

- could deliver in the future. 

3-18 




Review the first step in Phase 1 — Identify Products. 



• Explain the instructions for the activity. Have the attendees 
complete the first column of the form. Set the timer for 3 
minutes. 
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Once the three minutes have elapsed, ask the first 
one product they identified. Ask the group to go 
the criteria to see if that suggested product meets 
criteria. 



table to list 
over each of 
each of the 




• If it does, write it on the flip chart. 



• Go to the next table and repeat the process until you have at 
least one product from each table. 





• Put on Overhead 3-19. 



Customer Feedback 
Phase 1 : Prepare to Collect 

Continued 

Identify Customer(s) 

Who directly receives that product? 

• Working within your table, take 3 products 
that you have identified. 

• For each one, identify the customer(s) for 
that product. 

3-19 



• Review the second step in Phase 1 — Identify the Customer. 

The customer is the person who directly receives the product 
from the provider . 

• Instruct each table to take three products from their list and 
identify the customer(s) for each of these products. These can 
be listed next to each product in column 2 of Activity 
Worksheet 3-A. 
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A customer for one product may not be the customer for another 
product. This is why we list the products first 



• Set the timer for three minutes and tell the participants they 
have three minutes to identify customers for their products. 






Once the three minutes are up, ask each table to state 1 product 
(if it is a new one, review it against the 4 criteria). Then have 
them list all the customers of that product to whom they 
directly deliver the product. 



List the product named by the first table and each of its 
customers on the flip chart. 



Do a round robin of tables until you have at least one distinct 
product and all the customers of that product for each table. 



Put on Overhead 3-20. 



Customer Feedback 
Phase 1: Prepare to Collect 

Continued 

Identify What Customer Values 

• Product Outcome: What does the 
customer want as a result of the product? 

• Product Feature: “A satisfying product 
has...” 

3-20 



Review the third step in Phase 1 — Determine the Product 
Outcome (what the customer values about that product). 
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An easy question to get at the outcome/value of a product is to 
complete the sentence: “ A satisfying (product) RESULTS IN... ” 






Have each table use one of the products they listed in the last 
part of the exercise and choose one of the identified customers 
of that product. Utilizing that one customer of that one 
product, have the attendees complete the third column of 
Activity Worksheet 3-A under “what customers value about 
that product.” 




Again set the timer for 3 minutes. 



Once the three minutes are up, ask one table to identify the 
product they chose, the customer of that product, and how the 
customer would have completed the statement, “A satisfying 
(product) results in ” 



When you complete the phrase, “ a satisfying product RESULTS 
IN ...” you are identifying what the customer values about that 
product. 



• If tables are having trouble reporting the outcome, please help 
them rephrase their information in the form of an outcome. 




Record the outcomes identified by at least 2 tables on the flip 
chart. 



• Stay on Overhead 3-20. 

• Describe the 4 th step in Phase I — Identify Product Features. 



One way to determine what features of a product a customer 
values is to have the customer complete the following phrase: “ a 
satisfying product HAS. . . ” 
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• Give an example of some features from one of the 
customers/products that have been discussed already. 

There is not sufficient time today to identify features that 
customers value for all of your One-Stop products , but it is 
important to do so once you get back home . 



Customer Feedback 
Phase 1: Prepare to Collect 

Continued 

Identify Product Requirements 

What would satisfy the customer? 

Using one product and one customer of that 
product, list the adjectives the customer would 
use to describe a satisfying one. 



• Review the definition of a product requirement. 



A way to identify the requirements of the customer for a product 
is to have the customer complete the following statement, 

“A satifying (product) IS [one which ].... ” 

• Have each table take one of its products and one of that 
product’s customers. 

Fill in the sentence, “A satisfying (product) is....” the way your 
customer would. Use this information to complete the fourth 
column on Activity Worksheet 3-A. 




• Put on Overhead 3-21. 



3-21 
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• Set the timer for three minutes. 




• When the time is up, ask a table to report out by giving the 
name of the product, the customer group, and the requirements 
for a satisfying product. 



• Capture each table’s report out on the flip chart, using the 
columns as they have them listed on Activity Worksheet 3-A. 




Put on Overhead 3-22. 



Customer Feedback 
Phase 1 : Prepare to Collect 

Continued 

Identify Customer Standards 

How does the customer define his/her 
product requirement(s)? 

* Using your list of requirements, identify 
the customer’s definition of product 
requirements. 

3-22 



• Describe the definition of “standard.” 

• Have each table complete the last column of their form by 
taking the requirements that they established. For each 
requirement, have them identify the customer’s definition of the 
requirement (standard). 

• Set the timer for 3 minutes. 

• After the time is up, have one or two tables (as time permits) 
report out the standards they identified. 



• Discuss how important it is to have the customers identify the 
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features, values, requirements and standards. 

Today we have identified customer's values , requirements , and 
standards to understand what they might be. But it is not OUR 
ideas that are true customer requirements. We have to ask 
customers about their values , requirements, and standards. It is 
important to gather this input FROM THE CUSTOMER. 



• Remind attendees about the importance of getting detailed and 
complete customer information in the Customer/Provider 
activity at the beginning of the session. 




Collecting input from the customer is the second phase of the 
customer feedback cycle — Collecting Customer Feedback. 



• Put on Overhead 3-23. 



Customer Feedback 
Phase 2: Collect 




• Categories of Customer Feedback: 


-Qualitative 

-Quantitative 


3-23 



There are two types or categories of customer feedback: 
qualitative and quantitative. All customer feedback can be 
grouped into one or the other. 

• Put on Overhead 3-24 and 3-25. 

• Review the overhead points one at a time. 
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Customer Feedback 


Phase 2: Collect 




Continued 




Qualitative Data 


Examples of 


• Provides a clear and 


Qualitative Methods: 


precise statement of 


• Focus groups. 


the problem. 


• Advisory boards. 


* Identifies the "why." 


* Informal 


• Identifies what the 


conversations. 


customer may be 


• Management by 


feeling. 


walking around. 


• Data CANNOT be 




generalized. 


3-24 



Customer Feedback 
Phase 2: Collect 

Continued 




Quantitative Data 


Examples of 


• Attempts to quantify 


Quantitative 


attitudes and 


Methods: 


behaviors from the 


Paper and pencil 


population of 


surveys. 


interest. 


Mail surveys. 


• Identifies the "what", * 


Telephone surveys. 


"when", "where." 

• Draws on a sample 


Personal interviews. 


that represents the 
whole. 


3-25 




Put on Overhead 3-26. 



Customer Feedback 


Phase 2: Collect 

Continued 




Qualitative 


Quantitative 


• Describes belief. 


• Describes prevalence 


• No predetermined 


• Predetermined 


categories. 


categories 


• In-depth, small 


• Wide breadth, large 


sample of population. 


sample of population. 


• Subjective. 


• Objective 


• Interpretive - how and 


• Statistical - what and 


why. 


how many. 

3-26 
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• Ask the attendees to describe some of the methods they 
currently use to collect customer feedback. 



• Capture them on the flip chart. 



• Have them determine if they are qualitative or quantitative. 



Both qualitative and quantitative feedback can be collected in a 
proactive and a reactive manner. Proactive means that you 
solicit the feedback from the customer. Reactive means that you 
receive the feedback without prompting for it. 



• Using the methods on the flip chart, have the attendees identify 
which of the methods listed there are reactive and which are 
proactive. 



• Put on Overhead 3-27. 



Customer Feedback 
Phase 2: Collect 

Continued 




Proactive 


Reactive 


• Paper and pencil 


* Unsolicited 


surveys. 


communication. 


• Focus groups. 


Letters 


• Daily conversation. 


Phone calls 


• Alumni 


• Complaint logs. 


celebrations. 




• Follow up reports. 


3-27 



• Review the points on Overhead 3-27 with the work from the 
flip chart. 
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Put on Overhead 3-28. 



Customer Feedback 
Phase 2: Collect 

Continued 

What methods do you currently use to 
collect customer feedback? 

Qug.liMi.y-g? Quantitative? 

Reactive Reactive 

Proactive Proactive 

(Use Activity Worksheet 3-B to note your answers.) 

3-28 





Help attendees to locate Activity Worksheet 3-B (a form based 
on Overhead 3-28). 

At the tables have attendees list ALL the methods they use for 
collecting customer feedback and list it on Activity Worksheet 
3-B within the appropriate cell on the matrix. 

Put on Overhead 3-29. 



Customer Feedback 
Phase 2: Collect 

Continued 

Customer Research Considerations 

• Nothing is absolute 

• More complicated methods = greater 
chance of error 

• Choose the technique based on what 
you want to know 

3-29 
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• Review the overhead points one at a time. Make sure the 
attendees understand that the method an organization chooses 
to use in collecting customer feedback is based on the type of 
information that is needed. 

• If time allows, have attendees break into teams. Have some 
teams determine which of the quantitative methods seem to be 
the most complicated, while other teams determine which of the 
qualitative methods seem to be the most complicated. 

• Have the different teams share their conclusions. 



• Help attendees locate Activity Worksheet 3-C. Review the 
instructions for Activity Worksheet 3-C. 



Customer Feedback 
Phase 2: Collect 
Continued 

Working within your tables: 

• Review each scenario on Activity 
Worksheet 3-C. 

• Determine which collection method 
would be the most effective method to 
gather the customer data described. 

• Share with the other groups. 



• Have members at each table work together to complete the 
. exercise. 

• Set the timer for 5 minutes. 

• After 5 minutes, check to see how the different tables are 
progressing, and, if needed, give another 5 minutes. 




• Put on Overhead 3-30. 



3-30 
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• After the time is up, review the answers. 




We have now discussed the first two phases of the customer 
feedback cycle. 



The third phase will be discussed next. Who remembers what 
Phase 3 is? 



• Reward the person who says “analyze.” 

• State that there is a different approach to analyzing qualitative 
and quantitative data. 




• Put on Overhead 3-31. 



• Review the overhead points one at a time. 



Customer Feedback 
Phase 3: Analyze 

Qualitative Methods 

• Organize data. 

• Categorize/code data. 

• Interpret data. 

Known - confirmed or challenged? 

Suspected - confirmed or challenged? 

New information? 

• Write/deliver report. 

3-31 




Put on Overhead 3-32. 



Review the overhead points one at a time. 
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Customer Feedback 




Phase 3: Analyze 




Continued 




Quantitative Methods 




• Edit data. 




• Categorize/code data. 




* Enter data into research database. 




• Tabulate/summarize: 




- E.g., Tallies, percentages, averages 




• Determine variance. 




• Determine reliability/representation. 




• Display findings graphically. 


3-32 




• Note that you use similar but distinct analytic steps to analyze 
qualitative and quantitative data. To analyze qualitative data, 
you categorize, code, and interpret customer feedback. To 
analyze quantitative data, you categorize data, then use 
statistical analysis techniques to summarize and display findings. 

• Put on Overhead 3-33. 



Change is not what you 
know; it’s what you do with 
what you know. 



3-33 
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Read the overhead. 



Going through just the first 3 phases of the customer feedback 
cycle will not lead to improved customer satisfaction, unless you 
also do the fourth phase. 



Do you remember what is Phase 4? 



• Reward the person who says “utilize.” 




Put on Overhead 3-34. 



Customer Feedback 
Phase 4: Utilize 

• Every decision-from board policies to 
staff procedures-should be based upon 
customer data. 

• Use customer feedback to make 
individual daily improvements. What 
does this data tell me to do differently? 

• Implement customer feedback within a 
continuous improvement system. 

3-34 



• Review the overhead points one at a time. 



• Note that Board members should use customer data to establish 
policies that staff should utilize customer feedback to determine 
procedures. 



Feedback must be used to make daily improvements . Thus , you 
must be continuously collecting and analyzing feedback. This is 
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an ongoing cycle . It doesn ’t happen only one time but over and 
over again . This ongoing customer feedback cycle is what drives 
the continuous improvement process . 




Put on Overhead 3-35. 



Customer Feedback 
Phase 4: Utilize 

Continued 

Continuous Improvement Process 

• Study current situation (problem 
identification). 

• Determine root cause(s). 

• Choose a cause. 

• Determine solution(s)- choose one. 

• Test solution. 

• Implement solution. 

• Measure results- go back to beginning. 2-35 



• Review the overhead points one at a time. 



• Note that the “problem” may be something that is causing 
customer dissatisfaction, or, it could be something that is going 
OK today, but without improvement may cause customer 
dissatisfaction tomorrow. 



You don ’t have to improve ONLY those things that are problems. 




You must determine the root cause of a problem so that you are 
fixing the true problem and not just the symptom. 



• Note that there may be a variety of causes. It is important to 
choose the cause that is the most dissatisfying for the customer 
instead of one that only causes a little dissatisfaction. 
(Remember the 80/20 rule, which says that 80% of the 
problems result from 20% of the root causes.) 
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It is important to brainstorm a variety of possible solutions and 
then to review these possibilities to find the best. Do not just try 
the first solution that comes to mind. It may not be the best 
solution. 



It is important to test out the solution for “bugs” or 
unanticipated problems before undertaking full-scale 
implementation. 



• Note that testing the solution allows for refinement prior to full 
implementation. Part of testing the solution is gathering data to 
determine how well the solution worked. 



• Once one problem has been handled, it is important to go back 
to the beginning of the continuous improvement cycle and begin 
again with a new problem or area for improvement. 




• Put on Overhead 3-36. 



"The customer, in spirit and in 
flesh, must pervade the 
organization » 
every system in every 
department, every procedure, 
every measure, every meeting, 
every decision." 

Peters 

Thriving on Chaos 3-36 




• Read the quotation aloud. 

• Ask how One-Stops are currently demonstrating their 
commitment to this quote. 
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Capture the responses on the flip chart. 




• Put on Overhead 3-37. 



Customer Feedback 
Phase 4: Utilize 

Continued 

• Review the scenario described on 
Activity Worksheet 3-D. 

• Work with your group to determine what 
you would do to increase satisfaction 
and decrease dissatisfaction. 

• Share with the other groups. 

3-37 



• Help attendees to find Activity Worksheet 3-D. 




Review the activity instructions and ask for questions. Tables 
will work together. 



• Set the timer for 5 minutes. 



• After 5 minutes, check with the tables to see if the time is 
adequate. 



• Have each table report out. 
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Put on Overhead 3-38. 



Practicing the Steps to 
Customer Feedback 



• Focus Groups: one qualitative 
method of customer feedback 



3-38 



• State that since focus groups are not as widely used as surveys 
to gather customer feedback, we will be completing an exercise 
that gives some hands-on contact with focus groups. 



To elicit comprehensive customer feedback, a One-Stop system 
will need to collect feedback using both qualitative and 
quantitative methods. 



Who has conducted focus groups? 





Have individuals with focus group experience describe focus 
groups and how they have utilized them in their 
organizations/One-Stop systems. 

Put on Overhead 3-39. 
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Practicing the Steps to Customer 
Feedback - Focus Groups 

What Are Focus Groups? 

• The most effective way to collect qualitative 
data (the WHY of an issue). 

• Bring 8-12 people together. 

• Designated time and place. 

• Use a moderator (trained interviewer). 

• Loosely structured, free flowing dialogue 
focused on predetermined issues. 

• May be video taped/observed. 



• Review the overhead points one at a time. 



• Note the importance of having a skilled moderator for the focus 
group. One reason a skilled moderator is needed is to ensure 
that the discussion achieves a useful balance between “loosely 
structured/free flowing” and “focused.” 




• Put on Overhead 3-40. 



Practicing the Steps to Customer 
Feedback - Focus Groups 



• Determine 
Objectives 

• Review Objectives 
with Key Players 

• Determine Target 
Audience 

• Prepare Script 

• Identify/Recruit 
Participants 



• Conduct/Observe 
Focus Group 

• Analyze Focus 
Group 

• Present /Utilize Data 
Where are the 4 
phases within this 
list of steps? 
Where does each 
begin and end? 



• Review the overhead points one at a time. 



Compare the steps involved in conducting focus groups to the 
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four phases of the customer feedback cycle . Which of the focus 
group steps correspond to each of the four phases? 




Put on Overhead 3-41. 



Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 1 - Prepare to Collect 

• Determine objectives. 

Purpose? Information seeking? Issue/problem? Who wants? 
How will information be used? Only method? 

• Review objectives. 

Agree on purpose? 

• Determine target audience. 

• Prepare discussion guide. 

9 to 12 questions with 1 to 5 probes, 2 pages, structure using 
stages. 

3-41 



• Review the overhead points one at a time. 




Put on Overhead 3-42. 



Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 1 - Prepare to Collect, Continued 
• Identify/recruit participants: 

- Determine selection criteria 
(homogeneous, reflective of target, 
unfamiliar with each other, not likely to 
agree). 

- Develop potential list. 

- Develop screening list. 

-Screen, select, invite, confirm attendance. 

3-42 



• Review the overhead points one at a time. 
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Put on Overhead 3-43. 



Practicing the Steps to Customer 


Feedback - Focus 


Group 


Phase 2: Collect 




• Logistics 


• Observation 


- Comfortable, accessible. 


- Prepare note taking form 


neutral facility 


- Use tape recorder 


- Circle or u-shape room 


- Make seating chart 


arrangement 


- Organize notes around 


- Compensation 


questions 


• Moderator 


- Capture main points 


- Select 


- Use 10 effective questions 


- Skills required 






3-43 




• Review the overhead points one at a time. 

• Highlight the role of the moderator and the need for a highly 
skilled moderator to achieve the best results. 

• Highlight the role of the observers and how they need to take 
down quotable quotes, know where the questions are on the 
script, and capture the answers where they go instead of where 
the person may have responded. 

• Put on Overhead 3-44. 
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Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 2: Collect, Continued 
Focus Group Format 

• Introduction. 

Establish rapport, environment, session logistics. 

• Transition. 

Obtain overview, set stage for in-depth discussion. 

• In-Depth Investigation. 

Elicit details, enable participants to elaborate. 

• Closure. 

Clarify/alter, verify conclusions, thank participants, 
handle logistics. 3-44 



• Review the overhead points one at a time. 




Put on Overhead 3-45. 



Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 2: Collect 

Focus Group Role Play 

• Determine if you are an observer or 
participant. 

• Participants follow script handed out by 
trainer. 

• Observers take notes on participant 
comments using Activity Worksheet 3-E. 

3-45 




Ask for volunteers to be participants in a mock focus group. 

Once selected, hand out copies of the focus group script and 
have each participant select a person in the script to role play, 

Give the remaining attendees a copy of the observer form — 
Activity Worksheet 3-E. 



The role of observers is to listen and take notes while the role 
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play is happening . One useful practice for observers is to take 
down “ quotable quotes . ” 



• Conduct the role play focus group. 




Put on Overhead 3-46. 



Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 3: Analyze 

Using your notes, as a group, determine: 

• What requirements/standards the 
customers had for the One-Stop. 

• What products the customers identified. 

• What requirements the customers had 
for those products. 

• What standards they had for the 
requirements. 

3-46 




Have the attendees summarize the focus group findings, 
question by question. Capture notes on a flip chart. 



• Ask the observers first. Then augment with the focus group 
participants. 




Put on Overhead 3-47. 
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Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 4: Utilize 

Using the case study description found 
in Activity Worksheet 3-F: 

• Flow chart the customer flow for 
services that “help customers find jobs.” 

• Review your chart against the 
requirements/standards identified in the 
focus group summarized in the handout. 

• Revise/develop your customer flow to 
meet customer requirements. 3-47 




Activity Worksheet 3-F summarizes the results of a series of 
customer focus groups from a One-Stop in the state of 
Oklahoma > In this exercise you will analyze how customers flow 
through processes that “help customers find jobs . ” You will 
then use customer feedback to revise the customer flow to better 
meet customer requirements . 



• Have each group flow chart the current job referral process 
described in the example. 



• Set the timer for 5 minutes. Check with each group to make 
sure that they have had adequate time to complete the exercise. 



• When the time is up, have the groups review their current 
customer flow against the customer requirements that were 
stated in the case study. 



• Have the groups revise their flowchart, as needed, to more 
closely follow customer requirements and standards. 



• Set the timer for another 5 minutes. Check with each group to 
give adequate time. 



• Have each group report out on how they changed their job 
referral system to respond to customer feedback. Highlight the 
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key points from each group that show how they listened to and 
utilized customer requirements and standards from the case 
study. 



We have designed two “homework” assignments to help your 
local One-Stop partnership practice and apply the ideas 
presented in this session . 

Assignment #1 is designed to help you implement a process to 
collect and analyze customer feedback on an ongoing basis . 

Assignment #2 is designed to help you collect and analyze 
customer feedback on the One-Stop function “help customers 
find jobs. ” This information will be needed in training Module 
4 to guide your efforts to reengineer this function to improve 
customer satisfaction. 




3-48 
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Assignment 1: 

Develop Action Plan 

To Improve Customer Feedback 

• Review all of the materials of the session. 

• With your team members, review what 
customer feedback steps and phases are 
missing within your One-Stop. 

• Identify what you will do as first steps to 
revise your customer feedback process. 

• Write down your action plan. Have all 

team members sign. Share with a partner 
team. 3-49 



• Review the instructions for Assignment 1 . 



• If there is time during the training session, teams can spend five 
minutes brainstorming how they will approach Assignment 1 
and report out ideas to the whole group. 

• Emphasize the importance of setting measurable goals, specific 
time schedules, and assignments to make action plans happen. 



Assignment 2: 

Collect and Analyze Data on Customer 

Requirements 

• Review existing information on 
customer requirements for products that 
“help customers find jobs.” 

• Collect new information as needed from 
customer surveys and focus groups. 

• Summarize and analyze customer 
requirements. 

• Bring information to Module 4 session 
on Redesigning Customer Services3-50 



• Review the instructions for Assignment 2. 



• If the team will be proceeding with Module 4, develop an 
action plan for collecting customer feedback relevant to the 
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function “help customers find jobs ” 



• If the session on Module 4 will occur before any new data can 
be collected, have the team brainstorm what they think 
customers are concerned about and what customer 
requirements and standards would look like. 




The Tool Kit at the end of the Participant Workbook summarizes 
the process and continuous improvement tools that have been 
discussed in this and other sessions . 



• Put on Overhead 3-52. Review the tools covered in this 
session. 



Tools Referred to in 
Module 3 

• Using customer surveys and other 
quantitative feedback methods. 

• Using customer focus groups and other 
qualitative feedback methods. 

• Flow-charting the customer service 
process. 

• Comparing the current process to 
identified customer requirements. 

3-52 
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SPR SOCIAL POLICY RESEARCH ASSOCIATES 

Partnering for Quality Under the 
Workforce Investment Act: 

A TOOL KIT FOR 
ONE-STOP SYSTEM BUILDING 



Module 3 

COLLECTING AND USING CUSTOMER 
FEEDBACK 



Customer/Provider Activity 

Purpose 

♦ To learn the importance of using 
customer feedback to develop a product. 



3-2 



Customer/Provider Activity 

Instructions: 

♦ Find a partner-Designate one person as "A" 
and one person as "B." "A" s play the role of the 
customer; "B"s play the role of the provider. 

♦ The trainer will hand out picture to “A"s. 

♦ 'TV's will review a picture and develop a one- 
word description of what they want "B"s to draw 
for them. 

♦ M B"s have 2 minutes to draw the picture based 
on the one-word description. 

3-3 



Social Policy Research Associates 

O 

ERTC 



Page 3-1 



43 



Participant Workbook 

Module 3: Collecting and Using Customer Feedback 



Customer/Provider Activity 

Instructions, Continued 

♦ "A"s review drawing and determine if it meets 
their criteria. 

♦ 'TV's provide additional verbal feedback to ,, B"s 
and use the next 2 minutes to help them to 
create the picture they require. 

♦ Debrief the process. 

3-4 



Customer/Provider Activity 

Review Questions 



“B“s - When you were unable 
to check with your customer, 
did you provide the product 
they were expecting? 
u B“s - How did you feel/what 
did you think without 
information from your 
customer? Was it hard or 
easy to know what to draw? 
u B“s - How did you feel/what 
did you think when you worked 
with your customer? What 
was easy? What was hard? 



♦ “A“s - Did you get what you 
expected with the first picture? 

♦ "A“s - What difference did you 
find when you were able to 
communicate with the provider 
as they were drawing the 
picture? 

♦ As a customer (A), is your 

input important to get what you 
want and need? As a Provider 
(B), is the customer's input 
needed in order for you to 
know what the product should 
look tike? o e 



The Customer’s Perspective 

When one has no stake in the way things are, 
when one's needs or opinion are provided no 
forum, when one sees oneself as the object of 
unilateral actions, it takes no particular wisdom 
to suggest that one would rather be 
elsewhere. 

Seymour Sarason, 1990 
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Goals Clarification: 
Develop a System 
Perspective 




Get rid of: 


Create: 


C^Y” Products^) ^ 


“OUR” ) 


versus ^ 


Products J 


C^YOUR” Products^) 


3-7 



Learning Objectives 

♦ Learn why customer feedback is a 
foundation for One-Stop services. 

♦ Understand the process of customer 
feedback. 

♦ Understand the 4 phases of customer 
feedback. 

♦ Learn how to identify the product(s), 
customer(s), requirement(s), and standard(s) 
of the One-Stop system. 

3-8 



Learning Objectives, 

Continued 

♦ Understand the two types of customer 
feedback and the methods to collect each 
type. 

♦ Learn the methods to analyze customer 
feedback. 

♦ Explore how customer requirements impact 
program design, customer flow, staffing 
patterns, office layout. 

3-9 
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Learning Objectives, 

Continued 



♦ Leam how to create a team action plan to 
begin designing/refining your One Stop’s 
Customer Feedback Process. 
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Agenda 

♦ Introductions/Welcome 

♦ Why Use Customer Feedback to Design One- 
Stop Centers/Systems? 

♦ The Cycle of Customer Feedback 

♦ 4 Phases to the Customer Feedback Cycle 

♦ Prepare to Collect 

♦ Collect 

♦ Analyze 

♦ Utilize 

♦ Setting Goals for Improving Customer Services 

3-11 



The Customer Feedback 
Cycle 



ID Services/Products 
ID Customers of the 
Services/Products 
ID What Customer Values 
About Each 
Service/Product 
ID Customer Requirements 
(Attributes) for Each 
ID Product Features 
ID Customer Standards for 
the Requirements 
ID What to Measure 



♦ ID What Data to Collect 

Qualitative? 

Quantitative? 

♦ ID How to Measure - Which 
Statistical Process Control 
Tools 

—To Collect 
-To Analyze 

♦ Collect the Data 

♦ Analyze the Data 

— Meet/exceed Customer 
Standard? 

- Not Meet Customer Standard? 

♦ Make Improvements 
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4 Phases to the 
Customer Feedback Cycle 

♦ Prepare to Collect 

♦ Collect 

♦ Analyze 

♦ Utilize 

3-13 



Customer Feedback 
Phase 1 : Prepare to Collect 

FOCUSING THE PROCESS - {'THE MISSING 
PIECE") 

♦ Identify services and/or products. 

♦ Identify customers of each. 

♦ Identify what customer values about each. 

♦ Identify customer requirement(s)/attribute(s) 
for each. 

♦ Identify product features. 

3-14 



Customer Feedback 
Phase 1: Prepare to Collect 

Continued 

♦ Identify standards for each requirement. 

♦ Identify what to measure - what do you want 
to know? 

♦ Identify what data to collect. 

-- Qualitative? 

-- Quantitative? 

♦ Identify how to measure - which statistical 
process control tools to use to collect. 

3-15 
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Customer Feedback 
Phase 1: Prepare to Collect 

Continued 




Product Criteria 




♦ It must be a noun. 

♦ It must be made plural with an M s M 


or “es.” 


♦ It must be deliverable (you must be able 


to give it to someone). 

♦ It must be yours to give. 
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Customer Feedback 
Phase 1 : Prepare to Collect 

Continued 

♦ Activity Worksheet 3-A-ldentifying 
Customer Standards - will be used for 
an exercise that will ask you to : 

♦ Identify products. 

♦ Identify customers. 

♦ Identify product outcomes and features. 

♦ Identify customer requirements. 

♦ Define customer standards. 

3-17 



Customer Feedback 
Phase 1: Prepare to Collect 

Continued 

Identify Products 

♦ Working with your table, and using the 
criteria, identify products that your One- 
Stop: 

♦ delivers currently. 

♦ could deliver in the future. 

3-18 
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Customer Feedback 
Phase 1: Prepare to Collect 

Continued 

Identify Customer(s) 

Who directly receives that product? 

♦ Working within your table, take 3 products 
that you have identified. 

♦ For each one, identify the customer(s) for 
that product. 

3-19 



Customer Feedback 
Phase 1: Prepare to Collect 

Continued 

Identify What Customer Values 

♦ Product Outcome: What does the 
customer want as a result of the product? 
“A satisfying product results in...” 

♦ Product Feature: “A satisfying product 

has...” 3-20 



Customer Feedback 
Phase 1: Prepare to Collect 

Continued 

Identify Product Requirements 

What would satisfy the customer? 

♦ Using one product and one customer of that 
product, list the adjectives the customer would 
use to describe a satisfying one. 
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Customer Feedback 
Phase 1: Prepare to Collect 

Continued 

Identify Customer Standards 

How does the customer define his/her 
product requirement(s)? 

♦ Using your list of requirements, identify 
the customer’s definition of product 
requirements. 

3-22 



Customer Feedback 
Phase 2: Collect 




♦ Categories of Customer Feedback: 


♦ Qualitative 

♦ Quantitative 
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Customer Feedback 
Phase 2: Collect 

Continued 

Qualitative Data 

♦ Provides a clear and 
precise statement of 
the problem. 

♦ Identifies the “why." 

♦ Identifies what the 
customer may be 
feeling. 

♦ Data CANNOT be 
generalized. 



Examples of 
Qualitative Methods: 

♦ Focus groups. 

♦ Advisory boards. 

♦ Informal 
conversations. 

♦ Management by 
walking around. 

3-24 
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Customer Feedback 
Phase 2: Collect 

Continued 

Quantitative Data 

♦ Attempts to quantify 
attitudes and 
behaviors from the 
population of interest. 

♦ Identifies the “what”, 
“when”, “where.” 

♦ Draws on a sample 
that represents the 
whole. 



Examples of 
Quantitative Methods: 

♦ Paper and pencil 
surveys. 

♦ Mail surveys. 

♦ Telephone surveys. 

♦ Personal interviews. 
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Customer Feedback 
Phase 2: Collect 

Continued 



Qualitative 

♦ Describes belief. 

♦ No predetermined 
categories. 

♦ In-depth, small 
sample of population. 

♦ Subjective. 

♦ Interpretive - how and 
why. 



Quantitative 

♦ Describes prevalence 

♦ Predetermined 
categories 

♦ Wide breadth, large 
sample of population. 

♦ Objective 

♦ Statistical - what and 
how many. 
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Customer Feedback 




Phase 2: Collect 




Continued 




Proactive 


Reactive 


♦ Paper and pencil 


♦ Unsolicited 


surveys. 


communication. 


♦ Focus groups. 


Letters 


♦ Daily conversation. 


Phone calls 


♦ Alumni 


♦ Complaint logs. 


celebrations. 




♦ Follow up reports. 
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Customer Feedback 
Phase 2: Collect 
Continued 

What methods do you currently use to 
collect customer feedback? 

Qualitative? Q u a n t ita t iv e ? 

Reactive Reactive 



Proactive Proactive 

(Use Activity Worksheet 3-B to note your answers.) 
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Customer Feedback 
Phase 2: Collect 

Continued 

Customer Research Considerations 

♦ Nothing is absolute 

♦ More complicated methods = greater 
chance of error 

♦ Choose the technique based on what you 
want to know 

3-29 



Customer Feedback 
Phase 2: Collect 

Continued 

Working within your tables: 

♦ Review each scenario on Activity 
Worksheet 3-C. 

♦ Determine which collection method would 
be the most effective method to gather the 
customer data described. 

♦ Share with the other groups. 
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Customer Feedback 
Phase 3: Analyze 

Qualitative Methods 

♦ Organize data. 

♦ Categorize/code data. 

♦ Interpret data. 

Known - confirmed or challenged? 

Suspected - confirmed or challenged? 

New information? 

♦ Write/deliver report. 
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Customer Feedback 
Phase 3: Analyze 

Continued 

Quantitative Methods 

♦ Edit data. 

♦ Categorize/code data. 

♦ Enter data into research database. 

♦ Tabulate/summarize: 

♦ E g., Tallies, percentages, averages 

♦ Determine variance. 

♦ Determine reliabitity/representation. 

♦ Display findings graphically. 3.32 



Change is not what you 
know; it’s what you do with 
what you know. 
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Customer Feedback 
Phase 4: Utilize 

♦ Every decision should be based upon 
customer data, from board policies to staff 
procedures. 

♦ Use customer feedback to make individual 
daily improvements. What does this data 
tell me to do differently? 

♦ Implement customer feedback within a 
continuous improvement system. 
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Customer Feedback 
Phase 4: Utilize 

Continued 

Continuous Improvement Process 

♦ Study current situation (problem identification). 

♦ Determine root cause(s). 

♦ Choose a cause. 

♦ Determine solution(s)- choose one. 

♦ Test solution. 

♦ Implement solution. 

♦ Measure results- go back to beginning. 

3-35 



"The customer, in spirit and in 
flesh, must pervade the 
organization - 
every system in every 
department, every procedure, 
every measure, every meeting, 
every decision." 

Peters 

Thriving on Chaos 
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Customer Feedback 
Phase 4: Utilize 

Continued 



♦ Review the scenario described on Activity 
Worksheet 3-D. 

♦ Work with your group to determine what 
you would do to increase satisfaction and 
decrease dissatisfaction. 

♦ Share with the other groups. 

3-37 



Practicing the Steps to 
Customer Feedback 



♦Focus Groups: one qualitative 
method of customer feedback 



3-38 



Practicing the Steps to Customer 
Feedback - Focus Groups 

What Are Focus Groups? 

♦ The most effective way to collect qualitative 
data (the WHY of an issue). 

♦ Bring 8-12 people together. 

♦ Designated time and place. 

♦ Use a moderator (trained interviewer). 

♦ Loosely structured, free flowing dialogue 
focused on predetermined issues. 

♦ May be video taped/observed. o.3< 
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Practicing the Steps to Customer 
Feedback - Focus Groups 



♦ Determine Objectives 

♦ Review Objectives 
with Key Players 

♦ Determine Target 
Audience 

♦ Prepare Script 

♦ Identify/Recruit 
Participants 



♦ Conduct/Observe 
Focus Group 

♦ Analyze Focus Group 

♦ Present /Utilize Data 
Where are the 4 
phases within this 
list of steps? 

Where does each 
begin and end? 
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Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 1 - Prepare to Collect 

♦ Determine objectives. 

Purpose? Information seeking? Issue/problem? Who wants? 
How will information be used? Only method? 

♦ Review objectives. 

Agree on purpose? 

♦ Determine target audience. 

♦ Prepare discussion guide. 

9 to 12 questions with 1 to 5 probes, 2 pages, structure using stages. 
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Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 1 - Prepare to Collect , Continued 

♦ Identify/recruit participants: 

♦ Determine selection criteria (homogeneous, 
reflective of target, unfamiliar with each other, 
not likely to agree). 

♦ Develop potential list. 

♦ Develop screening list. 

♦ Screen, select, invite, confirm attendance. 

3-42 
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Practicing the Steps to Customer 
Feedback - Focus Group 



Phase 2: Collect 



♦ Logistics 

♦ Comfortable, accessible, 
neutral facility 

♦ Circle or u-shape room 
arrangement 

♦ Compensation 

♦ Moderator 

♦ Select 

♦ Skills required 



♦ Observation 

♦ Prepare note taking form 

♦ Use tape recorder 

♦ Make seating chart 

♦ Organize notes around 
questions 

♦ Capture main points 

♦ Use 10 effective questions 
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Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 2: Collect , Continued 

Focus Group Format 

♦ Introduction. 

Establish rapport, environment, session logistics. 

♦ Transition. 

Obtain overview, set stage for in-depth discussion. 

♦ In-Depth Investigation. 

Elicit details, enable participants to elaborate. 

♦ Closure. 

Clarify/alter, verify conclusions, thank participants, 
handle logistics. 3-44 



Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 2: Collect 

Focus Group Role Play 

♦ Determine if you are an observer or 
participant. 

♦ Participants follow script handed out by 
trainer. 

♦ Observers take notes on participant 
comments using Activity Worksheet 3-E. 

3-45 
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Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 3: Analyze 

Using your notes, as a group, determine: 

♦ What requirements/standards the 
customers had for the One-Stop. 

♦ What products the customers identified. 

♦ What requirements the customers had for 
those products. 

♦ What standards they had for the 
requirements. 
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Practicing the Steps to Customer 
Feedback - Focus Group 

Phase 4: Utilize 

Using the case study description found in 
Activity Worksheet 3-F: 

♦ Flow chart the customer flow for services 
that “help customers find jobs." 

♦ Review your chart against the 
requirements/standards identified in the 
focus group summarized in the handout. 

♦ Revise/develop your customer flow to 
meet customer requirements/stand aHff 
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Assignment 1 : 

Develop Action Plan 
To Improve Customer Feedback 

♦ Review all of the materials of the session. 

♦ With your team members, review what 
customer feedback steps and phases are 
missing within your One-Stop. 

♦ Identify what you will do as first steps to 
revise your customer feedback process. 

♦ Write down your action plan. Have all 

team members sign. Share with a partner 
team. 3-49 



Assignment 2: 

Collect and Analyze Data on 

Customer Requirements 

♦ Review existing information on customer 
requirements for products that “help 
customers find jobs.” 

♦ Collect new information as needed from 
customer surveys and focus groups. 

♦ Summarize and analyze customer 
requirements. 

♦ Bring information to Module 4 session on 
Redesigning Customer Services. 3-50 



Techniques and Tools 
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Tools Referred to in 
Module 3 

♦ Using customer surveys and other 
quantitative feedback methods. 

♦ Using customer focus groups and other 
qualitative feedback methods. 

♦ Flow-charting the customer service 
process. 

♦ Comparing the current process to 
identified customer requirements. 

3-52 



Social Policy Research Associates 



Page 3-18 





SPR 



Social Policy Research Associates 















Partnering for Quality under the 
Workforce Investment Act: 

A Tool Kit for 
One-Stop System Building 



Activity Worksheets 



Module 3 



L 









200 Middlefield Road, Suite 100 
Menlo Park, CA 94025 
Phone (650) 617-8625/Fax (650) 617-8630 



Activity Worksheets 

Module 3. Collecting and Using Customer Feedback 



ERIC 



W 

■o 

i_ 

(0 

2 1 

CO CO 

0 ) (0 
Q) L. 
.C <D 
t/> c 
£ 

i- o 

11 

><-> 

:Ii> 

4-1 ■- 



c 

© 

■O 



Possible Standard 
for 

Customer 

Requirement 
















Possible Customer 
Requirement for 
Product/Service 
















What Customer Values 
about That 
Product/Service: 
Outcome 
















Customer of That 
Product/Service (Person 
Who Directly Receives That 
Product/Service) 
















Product/Service That 
Your Organization 
Provides 

















CO 



< 

I 

CO 

Q) 

(D 

jC 

CO 



■> 

o 

< 



CD 



ERIC 



o 

03 

_Q 

•o 

CD 

03 



03 

E 

o 

GO 

3 

O 

O) 

c 

CO 

D 

*o 

c 

03 



"O 

CD (/) 
3 T3 

I! 

o ra 

°.W 

< j- 
■ 0) 
CO c 

o o 
0> w 



0) 

"5 

CJ 

II 

+■* 

is 

O 2 

< 



*0 
O o 
< ^ 



Possible Standard 
for 

Customer 

Requirement 
















Possible Customer 
Requirement for 
Product/Service 
















What Customer Values 
about That 
Product/Service 
















Customer of That 
Product/Service (Person 
Who Directly Receives That 
Product/Service) 
















Product/Service That 
Your Organization 
Provides 

















O 



CO 



CD 

CD 

_c 

CO 



o 

§ 



>% 

;> 

o 

< 



CD 

CD 



Activity Worksheets 

Module 3. Collecting and Using Customer Feedback 



Activity Worksheet 3-B 
What Customer Feedback Methods Do You 
Currently Use? 





Qualitative 


Quantitative 


Reactive 






Proactive 
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Activity Worksheet 3-C 
Choosing Customer Feedback Methods 



Your organization is interested in knowing the following 
information. For each type of information describe the method 
you would recommend to gather it and why. 

• What percentage of welfare mothers believe that child 
care should be provided at the One Stop? 



• The number of youth that completed training in Program 
Year 1997 compared to the number of adults and older 
workers. 



• What products/services should be delivered in the One 
Stop? 



• How satisfied are the customers using your services? 



• What are the requirements/standards for the top 3 
products/services requested by your customers? 
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Activity Worksheet 3-D 
(Version 1) 

Utilizing Customer Feedback 

Survey results state that 95% of your 
customers are dissatisfied with your 
services because they had to wait for 
more than 30 minutes after arrival before 
being able to meet with a staff person. 
Your team was given the responsibility to 
decrease customer dissatisfaction and 
increase customer satisfaction. What do 
you do? 
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Activity Worksheet 3-D 
(Version 2) 

Utilizing Customer Feedback 



Survey results state that two out of 
every three job seeker plus customers 
who use the resource room would use it 
more if you were open nights and 
weekends. Your team was given the 
responsibility of increasing customer 
satisfaction. What do you do? 
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Activity Worksheet 3-D 
(Version 3) 

Utilizing Customer Feedback 



Survey results state that employer 
customers are dissatisfied with your 
services because the response time for 
referring applicants is two weeks. Your 
team was given the responsibility to 
decrease customer dissatisfaction and 
increase customer satisfaction. What 
do you do? 
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Activity Worksheet 3-D 
(Version 4) 

Utilizing Customer Feedback 



Survey results state that 75% of your 
employer customers are dissatisfied 
with your services because they have to 
explain their needs to multiple workforce 
development representatives/agencies 
and no one seems to have a grasp on 
the total scope of available services. 

The employer customers say they want 
a single point of contact to be satisfied. 
Your team was given the responsibility 
to decrease customer dissatisfaction 
and increase customer satisfaction. 
What do you do? 
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Activity Worksheet 3-E 
Roleplay Script for Employer Focus Group 

Topic: Job Referral Services 



I. INTRODUCTION- 10 minutes 

Welcome! My name is and I will be your moderator for this focus group. 

Thank you for agreeing to be a part of our focus group. Your participation today will 
provide us with the opportunity to gather information and feedback from you in order to 

increase your satisfaction with the job referral services offered by 

Workforce Development. 

Although some of you may know each other, we would like each of you to introduce 
yourselves, Please tell us our name, the company you represent, and your job title. This 

will help me as a moderator of the focus group and our observers , 

,and, get to know you.. 

Workforce Development is transitioning and experiencing change. Your 

input will be valuable in assisting us to better meet the needs of our employer 
customer. We provide job referral and placement services to employers in the 
surrounding six counties ( A,B,C,X,Y,Z.) Our purpose is to provide quality job 
applicants in a timely manner. We attempt to meet your needs through the use of our 
computerized job matching system. This system matches applicant skills, experience 
and education to specific job requirements. In order for this process to work more 
efficiently, we need specific information from both the employer and the prospective job 
applicant. We will focus on what a satisfying recruitment service is, how useful our job 
referral service have been in the past, what skills/requirements you need from qualified 
job applicants and what improvements can be made to the current job referral process. 

We are conducting five additional focus groups like this one so that we can design a 
system that you, the customers say they want, need and will use. We have developed 
a list of questions addressing issues we feel are important to the success of he referral 
and placement system and meeting your need to find skilled workers. 
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During the next 90 minutes, we will be having an open dialogue regarding the 

workforce development referral and placement process. All of you have today have 

used the services of one or more of the agencies within workforce 

development. We will be asking questions about when you have used those agencies 
and when you have used other referral and recruitment agencies. There are no right or 
wrong answers. Each of you has equally valuable information and thoughts that are 
very interested in hearing. I would encourage you to be as open and honest with during 
the meeting as you possibly can. 

The purpose of our observers is to record what you say. We don’t want to miss any of 
your ideas. In addition to taking notes, we will be videotaping this session. I want to 
assure you that the information gathered at this meeting will be kept confidential and 
anonymous. The tapes will not be used to identify who said what, but to record all the 
ideas from the discussion. 

Because our time is limited during this session, my role as moderator is to make 
certain that all topics are covered and to move the discussion along, while ensuring 
that everyone has a chance to participate in the discussion. As moderator, I want to 
review the guidelines for participating in focus group: 

Give priority to those who have not spoken 
Speak briefly and often 

Speak one at a time. ..not on top of each other 
Be open and honest 

Be specific.. .talk from your own experiences 

Listen without interruption 

Present problems.. .focus on solutions 

It is okay to disagree.. .but not be disagreeable 

There are no right or wrong answers.. .all responses are valid 

“Piggy-back” on (augment) other’s answers 

Again, the focus group is scheduled to last approximately 90 minutes. Are there any 
questions about what we are going to do today or about anything that I have said? 
Okay, let’s turn on the tape and begin our discussion. 
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Activity Worksheet 3-E 
Script for Employer Focus Group 
Continued 

• It is okay to disagree, but not be disagreeable. 

• There are no right or wrong answers. All responses are valid. 

• “Piggy-back” on (augment) others’ answers. 

Again, the focus group is scheduled to last approximately 90 minutes. Are 
there any questions about what we are going to do today or about anything 
that I have said? Okay, let’s turn on the tape and begin our discussion. 

MODERATOR’S QUESTIONS FOR 
EMPLOYER FOCUS GROUP ON JOB REFERRAL PROCESS 

Note: An Observer Note-Taking Form (With Space Between Questions) 

Follows the Script 

I. REQUIREMENTS FOR A SATISFYING RECRUITMENT RESOURCE 
(20 minutes) 

Contextual statement: The focus of our first area of discussion is on finding out what 

you look for when choosing a recruitment resource. What is 
important to you in how you receive those services? 

Primary Question: A satisfying recruitment resource IS 

REQUIREMENT: STANDARD - 

(What does that mean to you?) 



Primary Question: Which are the most important of these requirements? 



Primary Question: Which are the least Important of these requirements? 
Primary Question: A satisfying recruitment resource has 
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Activity Worksheet 3-E 
Script for Employer Focus Group 

Continued 

I. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 
(20 minutes) 

Contextual statement: The focus of the second area of discussion is to identify your 

past experiences with job referral services provided by the 
workforce development system agencies we discussed earlier. 

Primary Question: As you think back on those placement service experiences that you 
had with those agencies, what was satisfying to you? 



II. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 

Continued 

What was not satisfying? 

What could have made it more satisfying? 

III. SKILL REQUIREMENTS - 20 minutes 

Contextual Statement: The focus of the third area of discussion is on finding out what 

skills/qualities you require of the job applicant referrals we make 
to you that would lead to a better match, less employee turnover 
and more productivity. 

Primary Question: What are the skills/qualities you are looking for when hiring 

an individual? 

Skills Further Describe: 



Primary Question: What are the skills that are the most important to you? 

Primary Question: What are the skills that are least important to you? 
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Activity Worksheet 3-E 
Script for Employer Focus Group 

Continued 



IV. WRAP UP COMMENTS 

1. In summary - here are the important items that I learned from you about 

A. The requirements of a recruitmentljob referral agency are 

B. What you liked about the experiences with our agencies is 

C. What you disliked about your experiences with us is 

D. The skills! qualities you want in a qualified employee are 

2. What did I miss? 

3. Before we wrap up, I would like to open the discussion for you to bring up any 
other issues that you would like to discuss on these topics. 

4. Here’s what we will do with the information we have gathered today: 

• Take information from all 6 focus groups 

• Analyze it 

• Make decisions on designing referral processes, how referrals will be made and 
how you will receive them. 

• Feed back this information to you in a report as well as in our changed 
behaviors. 

Thank you again for helping us to design a referral and placement system that will be 
useful to you. 
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MODERATORS QUESTION FOR 

EMPLOYER FOCUS ON JOB RFERRAL PROCESS 

I. SATISFYING RECRUITMENT RESOURCE - 20 minutes 



Contextual statement: 


The focus of our first area of discussion is on finding 
out what you look for when choosing a recruitment 
re source... what is important to you in how you receive 
those services. 


Primary Question: 


A satisfying recruitment resource IS 


REQUIREMENT: 


STANDARD-What does that mean to you? 


TIMELY 


WITHIN 24 HOURS FOR MOST JOBS 
2-3 weeks max for professional ones 
1-2 weeks max for seasonal 


EASY TO ACCESS 


NO PAPERWORK HASSLES 


CONVENIENT 


ABLE TO ACCESS HOW/WHEN AND 
WHERE 1 WANT 

CAN BE DONE THRU FAX/INTERNET 


EXPANSIVE 


HAVE A LARGE POOL OF 
JOB APPLICANTS 


PRE-SCREENED 


MEETS MY JOB QUALIFICATIONS 


Primary Question: 

PRE-SCREENED 

PRE-SCREENED 

PRE-SCREENED 

EASY TO ACCESS 

CONVENIENT 

CONVENIENT 


Which are the most important of these requirements? 


Primary Question: 


Which are the least important of these requirements? 



LARGE POOL OF APPLICANTS 
LARGE POOL OF APPLICANTS 
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Primary Question: A satisfying recruitment resource HAS 

A LARGE POOL OF APPLICANTS THAT MEET MY SKILLS NEEDS 
A LARGE POOL OF APPLICANTS THAT MEET MY SKILLS NEEDS 

FRIENDLY STAFF 

KNOWLEDGEABLE STAFF (staff that know me and my needs, that know how 
to screen to fit my particular organization’s culture) 

INTERNET ACCESS-1 am able to input my job order on the internet-without 
having to call your office 

INTERNET ACCESS 
INTERNET ACCESS 

STAFF WHO ARE ABLE TO PRE-SCREEN FOR I-9S, LEGAL PROBLEMS OF 
THE APPLICANT, TRUTH ABOUT EDUCATION CERTIFICATIONS AND 
DEGREES, PREVIOUS POOR WORK HISTORY OF THE JOB APPLICANTS 



II. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 

REFERRAL SERVICES-20 minutes 

Contextual Statement: The focus of this second area is to identify your past 

experiences with job referral services provided by the 
workforce development system agencies we 
discussed earlier. 

Primary Question: As you think back on those placement services 

experiences that you had with those agencies, what was satisfying to you? 

FRIENDLY STAFF 
FRIENDLY STAFF 
FRIENDLY STAFF 

KNOWLEDGEABLE STAFF 
KNOWLEDGEABLE STAFF 



Who was NOT satisfying? 

I HAD TO WAIT TOO LONG FOR THE REFERRALS-4 DAYS IS TO LONG 
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I HAD TO WAIT TO LONG FOR THE REFERRALS 

ALL OF THE JOB APPLICANTS WERE NOT ADEQUATELY SCREENED-SOME 
OF THEM SAID THAT THEY HAD SCHOOLING THAT THEY DID NOT HAVE 

YOUR HOURS OF OPERATION ARE NOT EXPANSIVE ENOUGH-1 NEED TO BE 
ABLE TO ACCESS YOU ON SATURDAYS ONR AFTER 6 PM DURING THE 
WEEK 

What could have made it more satisfying? 

QUICKER TURNAROUND ON REFERRALS 

ABLE TO ACCESS WHEN I NEED IT 

III. SKILL REQUIREMENTS 20-minutes 



Contextual Statement: 


The focus of the third area of discussion is on finding 
out want skills/qualities you require of the job 
applicant referrals we make to you that would lead to 
a better match, less employee turnover and more 
productivity. 


Primary Question: 


What are the skills/qualities you are looking for when 
hiring an individual? 


skills: 


further describe: 


COMPUTER SKILLS 
COMPUTOR SKILLS 


KEYBOARD/WORD PROCESSING 
WORD PROCESSING 


DESIRE TO WORK 
FOLLOW THROUGH 
RELIABLE 


THERE EVERY DAY, ON TIME 


RELIABLE 


SCHEDULE DR. APPOINTMENTS OFF WORK 


TEAM SKILLS 


ABLE TO WORK WITH TEAMS TO GET WORK 
DONE 


PROBLEM SOLVING 


DON’T HAVE TO ALWAYS ASK WHAT TO DO 
INTERDEPENDENT 


TEAM SKILLS 
DESIRE TO WORK 


CAN WORK WITH OTHERS 

COMES READY TO WORK-DOESN’T BRING 

HOME PROBLEMS WITH THEM 
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GOOD BASIC SKILLS MATH FOR SPC, CAN READ INSTRUCTIONS 

FOCUSED ON THE CUSTOMER 

GOOD BASIC SKILLS I CAN TEACH THEM THE JOB SKILLS IF THEY 

HAVE THE READING AND MATH SKILLS AMD 
SKILLS THSAT HELP THEM LEARN 

Primary Question: What are the skills that are the most important to 

you? 

COMPUTER SKILLS, BASIC SKILLS 
COMPUTER SKILLS RELIABLE, WILLING TO WORK 
BASIC SKILLS, WILLING TO WORK 

Primary Question: What are the skills that are important to you? 

NONE 

ALL ARE IMPORTANT 
ALL ARE IMPORTANT 
IV. WRAP UP 20 minutes 

1. In summary-here are the important items that I learned from you about 

A. The requirements of a recruitmentljob referral agency are... .. . 

B. What you liked about the experiences with our agencies is. . . 

C. What you disliked about your experiences with us is... 

D. The skills/qualities you want in a qualified employee are... 

2. What did I miss? 

EVERYTHING WAS COVERED 

YOU CAPTURED EVERYTHING 

3. Before we wrap up, I would like to open the discussion for you to bring up 
any other issues that you would like to discuss on these topics. 

WILL YOU BE ABLE TO STRAT SERVE PEOPLE THAT ARE NOT TARGETTED? 
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NT IS HARD FOR ME TO FING PEOPLE WITH THE SKILLS I NEED AND I WANT 
YOU TO HAVE THE LARGEST POOL OF PEOPLE TO WORK WITH AS 
POSSIBLE 

I WOULD LOVE FOR YOUR AGENCEY TO LESSEN THE RISK FOR MY 
ORGANIZATION TO HIRE PEOPLE-COULD YOU REFER PEOPLE TO ME THAT 
YOU WOULD PAY THEIR WAGES DURING A TRYOUT PERIOD OF 3 MONTHS. I 
WOULD BE WILLING TO PAY YOU, WITH A SMALL ADMINISTRATIVE FEE... IF 
YOU COULD LESSON MY LIABLITY ON NEW WORKERS. I WOULD THEN BE 
ABLE, AFTER THE TRYOUT PERIOD 

• 4. Here’s what we will do with the information we have gathered 

today... 

• Take information from 6 focus groups 

• Analyze it 

• Make decisions on designing referral processes, how referrals will be 
made and how will you receive them. 

• Feed back this infromation to you in a report as well in our changed 
behaviors. 



Thank you again for helping us to design a referral and placement system that will be 
useful to you. 
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Activity Worksheet 3-E 
Note-Taking Form 

for Employer Focus Group on Job Referral Services 

ATTENDEES: 



II. REQUIREMENTS FOR A SATISFYING RECRUITMENT RESOURCE 
(20 minutes) 

Contextual statement: The focus of our first area of discussion is on finding out what 

you look for when choosing a recruitment resource. What is 
important to you in how you receive those services? 

Primary Question: A satisfying recruitment resource IS 



REQUIREMENT: STANDARD - 

(What does that mean to you?) 
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Activity Worksheet 3-E 
Observer Note Taking Form 

Continued 

II. REQUIREMENTS FOR A SATISFYING RECRUITMENT RESOURCE 

Continued 

Primary Question: A satisfying recruitment resource IS 

Continued 

Most Important? 



Least Important? 



Primary Question: A satisfying recruitment resource has 
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Activity Worksheet 3-E 
Observer Note Taking Form 

Continued 

III. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 
(20 minutes) 

Contextual statement: The focus of the second area of discussion is to identify your 

past experiences with job referral services provided by the 
workforce development system agencies we discussed earlier. 



Primary Question: As you think back on those placement service experiences that you 
had with those agencies, what was satisfying to you? 



What was not satisfying? 



Activity Worksheet 3-E 



13 



Activity Worksheets 

Module 3. Collecting and Using Customer Feedback 

Activity Worksheet 3-E 
Observer Note-Taking Form 

Continued 

III. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 

Continued 

Primary Question: As you think back on those placement service experiences that you 
had with those agencies, what was satisfying to you? 

Continued , 



What could have made it more satisfying? 



III. SKILL REQUIREMENTS - 20 minutes 

Contextual Statement: The focus of the third area of discussion is on finding out what 

skills/qualities you require of the job applicant referrals we make 
to you that would lead to a better match, less employee turnover 
and more productivity. 

Primary Question: What are the skills/qualities you are looking for when hiring 

an individual? 

Skills Further Describe: 
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Primary Question: 



Primary Question: 



Activity Worksheet 3-E 
Observer Note-taking Form 

Continued 

What are the skills that are the most important to you? 



What are the skills that are least important to you? 
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Activity Worksheet 3-E 
Observer Note-taking Form 

Continued 



V. WRAP UP COMMENTS 



Missed Items: 



Additional items not previously covered: 




Activity Worksheet 3-E 



16 



94 



Activity Worksheets 

Module 3. Collecting and Using Customer Feedback 

Activity Worksheet 3-E 

Observer Note-taking Form 
for Employer Focus Group on Job Referral Services 



ATTENDEES: 



III. REQUIREMENTS FOR A SATISFYING RECRUITMENT RESOURCE 
(20 minutes) 

Contextual statement: The focus of our first area of discussion is on finding out what 

you look for when choosing a recruitment resource. What is 
important to you in how you receive those services? 

Primary Question: A satisfying recruitment resource IS 



REQUIREMENT: STANDARD - 

(What does that mean to you?) 
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Activity Worksheet 3-E 

Observer Note-taking Form 

Continued 

IV. REQUIREMENTS FOR A SATISFYING RECRUITMENT RESOURCE 

Continued 

Primary Question: A satisfying recruitment resource IS 

Continued 

Most Important? 



Least Important? 



Primary Question: A satisfying recruitment resource has 
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Activity Worksheet 3-E 

Observer Note-taking Form 

Continued 

V. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 
(20 minutes) 

Contextual statement: The focus of the second area of discussion is to identify your 

past experiences with job referral services provided by the 
workforce development system agencies we discussed earlier. 

Primary Question: As you think back on those placement service experiences that you 
had with those agencies, what was satisfying to you? 



What was not satisfying? 
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Activity Worksheet 3-E, Continued 

IV. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 

Continued 

Primary Question: As you think back on those placement service experiences that you 
had with those agencies, what was satisfying to you? 

Continued 



What could have made it more satisfying? 



III. SKILL REQUIREMENTS - 20 minutes 

Contextual Statement: The focus of the third area of discussion is on finding out what 

skills/qualities you require of the job applicant referrals we make 
to you that would lead to a better match, less employee turnover 
and more productivity. 

Primary Question: What are the skills/qualities you are looking for when hiring 

an individual? 

Skills Further Describe: 
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Primary Question: 



Primary Question: 
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Observer Note-taking Form 

Continued 

What are the skills that are the most important to you? 



What are the skills that are least important to you? 
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Activity Worksheet 3-E 

Observer Note-taking Form 

Continued 

VI. WRAP UP COMMENTS 



Missed Items: 



Additional items not previously covered: 
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Activity Worksheet 3-F 
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Participant Workbook 

Module 3. Collecting and Using Customer Feedback 

Activity Worksheet 3-E 
Script for Employer Focus Group 
on Job Referral Services 

INTRODUCTION -10 minutes 

Welcome! My name is and I will be your moderator for this 

focus group. 

Thank you for agreeing to be a part of our focus group. Your participation 
today will provide us with the opportunity to gather information and 
feedback from you in order to increase your satisfaction with the job referral 
services offered by Workforce Development. 

Although some of you may know each other, we would like each of you to 
introduce yourselves. Please tell us your name, the company you 
represent, and your job title. This will help me as a moderator of the focus 

group and our observers , , and 

get to know you. 

Workforce Development is transitioning and experiencing 

change. Your input will be valuable in assisting us to better meet the needs 
of our employer customer. We provide job referral and placement services 
to employers in the surrounding six counties (A, B, C, X, Y, Z.) Our purpose 
is to provide quality job applicants in a timely manner. We attempt to meet 
your needs through the use of our computerized job matching system. This 
system matches applicant skills, experience and education to specific job 
requirements. In order for this process to work more efficiently, we need 
specific information from both the employer and the prospective job 
applicant. We will focus on what a satisfying recruitment service is, how 
useful our job referral services have been in the past, what 
skills/requirements you need from qualified job applicants and what 
improvements can be made to the current job referral process. 

We are conducting five additional focus groups like this one so that we can 
design a system that you, the customers say they want, need and will use. 
We have developed a list of questions addressing issues we feel are 
important to the success of the referral and placement system and meeting 
your need to find skilled workers. 
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Activity Worksheet 3-E 
Script for Employer Focus Group 

Continued 

During the next 90 minutes, we will be having an open dialogue regarding 

the workforce development referral and placement process. All 

of you here today have used the services of one or more of the agencies 

within workforce development. We will be asking questions 

about when you have used those agencies and when you have used other 
referral and recruitment agencies. There are no right or wrong answers. 
Each of you has equally valuable information and thoughts that we are very 
interested in hearing. I would encourage you to be as open and honest 
with during the meeting as you possibly can. 

The purpose of our observers is to record what you say. We don’t want to 
miss any of your ideas. In addition to taking notes, we will be videotaping 
this session. I want to assure you that all the information gathered at this 
meeting will be kept confidential and anonymous. The tapes will not be 
used to identify who said what, but to record all the ideas from the 
discussion. 

Because our time is limited during this session, my role as moderator is to 
make certain that all topics are covered and to move the discussion along, 
while ensuring that everyone has a chance to participate in the discussion. 
As moderator, I want to review the guidelines for participating in a focus 
group: 

• Give priority to those who have not spoken. 

• Speak briefly and often. 

• Speak one at a time, not on top of each other. 

• Be open and honest. 

• Be specific. Talk from your own experiences. 

• Listen without interruption. 

• Present problems. Focus on solutions. 
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Activity Worksheet 3-E 
Script for Employer Focus Group 
Continued 

• It is okay to disagree, but not be disagreeable. 

• There are no right or wrong answers. All responses are valid. 

• “Piggy-back” on (augment) others’ answers. 

Again, the focus group is scheduled to last approximately 90 minutes. Are 
there any questions about what we are going to do today or about anything 
that I have said? Okay, let’s turn on the tape and begin our discussion. 

MODERATOR’S QUESTIONS FOR 
EMPLOYER FOCUS GROUP ON JOB REFERRAL PROCESS 



Note: An Observer Note-Taking Form (With Space Between Questions) 
Follows the Script 

I. REQUIREMENTS FOR A SATISFYING RECRUITMENT RESOURCE 
(20 minutes) 

Contextual statement: The focus of our first area of discussion is on finding out what 

you look for when choosing a recruitment resource. What is 
important to you in how you receive those services? 

Primary Question: A satisfying recruitment resource IS 

REQUIREMENT: STANDARD - 

(What does that mean to you?) 



Primary Question: Which are the most important of these requirements? 



Primary Question: Which are the least Important of these requirements? 
Primary Question: A satisfying recruitment resource has 
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Activity Worksheet 3-E 
Script for Employer Focus Group 

Continued 

II. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 
(20 minutes) 

Contextual statement: The focus of the second area of discussion is to identify your 

past experiences with job referral services provided by the 
workforce development system agencies we discussed earlier. 

Primary Question: As you think back on those placement service experiences that you 
had with those agencies, what was satisfying to you? 



II. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 

Continued 



What was not satisfying? 

What could have made it more satisfying? 

III. SKILL REQUIREMENTS - 20 minutes 

Contextual Statement: The focus of the third area of discussion is on finding out what 

skills/qualities you require of the job applicant referrals we make 
to you that would lead to a better match, less employee turnover 
and more productivity. 

Primary Question: What are the skills/qualities you are looking for when hiring 

an individual? 

Skills Further Describe: 



Primary Question: What are the skills that are the most important to you? 



Primary Question: What are the skills that are least important to you? 
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Activity Worksheet 3-E 
Script for Employer Focus Group 

Continued 



IV. WRAP UP COMMENTS 



1. In summary - here are the important items that I learned from you about 

A. The requirements of a recruitment/job referral agency are 

B. What you liked about the experiences with our agencies is 

C. What you disliked about your experiences with us is 

D. The skills/qualities you want in a qualified employee are 

2. What did I miss? 

3. Before we wrap up, I would like to open the discussion for you to bring up any 
other issues that you would like to discuss on these topics. 

4. Here’s what we will do with the information we have gathered today: 

• Take information from all 6 focus groups * 

• Analyze it 

• Make decisions on designing referral processes, how referrals will be 
made and how you will receive them. 

• Feed back this information to you in a report as well as in our changed 
behaviors. 

Thank you again for helping us to design a referral and placement system that will be 
useful to you. 
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Activity Worksheet 3-E 
Note-Taking Form 

for Employer Focus Group on Job Referral Services 

ATTENDEES: 



II. REQUIREMENTS FOR A SATISFYING RECRUITMENT RESOURCE 
(20 minutes) 

Contextual statement: The focus of our first area of discussion is on finding out what 

you look for when choosing a recruitment resource. What is 
important to you in how you receive those services? 

Primary Question: A satisfying recruitment resource IS 



REQUIREMENT: STANDARD - 

(What does that mean to you?) 
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Activity Worksheet 3-E 
Observer Note Taking Form 

Continued 

III. REQUIREMENTS FOR A SATISFYING RECRUITMENT RESOURCE 

Continued 

Primary Question: A satisfying recruitment resource IS 

Continued 

Most Important? 



Least Important? 



Primary Question: A satisfying recruitment resource has 
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Activity Worksheet 3-E 
Observer Note Taking Form 

Continued 

IV. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 
(20 minutes) 

Contextual statement: The focus of the second area of discussion is to identify your 

past experiences with job referral services provided by the 
workforce development system agencies we discussed earlier. 

Primary Question: As you think back on those placement service experiences that you 
had with those agencies, what was satisfying to you? 



What was not satisfying? 
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Activity Worksheet 3-E 
Observer Note-Taking Form 

Continued 

III. EXPERIENCES WITH WORKFORCE DEVELOPMENT AGENCY JOB 
REFERRAL SERVICES 

Continued 

Primary Question: As you think back on those placement service experiences that you 
had with those agencies, what was satisfying to you? 

Continued 



What could have made it more satisfying? 



III. SKILL REQUIREMENTS - 20 minutes 

Contextual Statement: The focus of the third area of discussion is on finding out what 

skills/qualities you require of the job applicant referrals we make 
to you that would lead to a better match, less employee turnover 
and more productivity. 

Primary Question: What are the skills/qualities you are looking for when hiring 

an individual? 

Skills Further Describe: 
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Primary Question: 



Primary Question: 



0 
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Observer Note-taking Form 

Continued 

What are the skills that are the most important to you? 



What are the skills that are least important to you? 
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Activity Worksheet 3-E 
Observer Note-taking Form 

Continued 



V. WRAP UP COMMENTS 



Missed Items: 



Additional items not previously covered: 
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Case Study Example 

Module 3. Collecting and Using Customer 
Feedback 

Central Texas Private Industry Council 

In Central Texas, a “bottom-up” approach was used to develop a customer-driven 
service delivery system 1 . First, a customer satisfaction survey was used to establish 
the baseline by which to measure expected improvements. When reviewing the survey 
results, it became clear that “while we knew how customers felt about what mattered to 
us, we were no closer to understanding how customers felt about what mattered to 
them”. Central Texas then moved “Beyond the Survey” and selected a multiple focus 
group format as stage two in their efforts to facilitate a customer communication 
process. The focus group plan included four job seeker focus groups and two 
employer focus groups, followed by a front-line staff “workshop” . The front-line staff 
workgroup included representatives from all of the One-Stop partners, who were tasked 
with analyzing the focus group responses and recommending service improvements. 

Focus group participants were solicited through multiple methods of marketing, 
including news releases, display advertising, public service announcements and 
Chamber of Commerce newsletters. Each of the focus groups was provided with a 
“Customer Information Packet”, which included an individualized “thank-you” letter 
and informational brochures about One-Stop services. The focus group questions were 
carefully crafted by staff (“Don’t ask about things that you can’t fix!” advises strategic 
planner and project director Linda Angel). The University of Central Texas Research 
Institute provided skilled “neutral” moderators. 

A summary of the key points brought out through the focus groups was 
developed by the facilitators and presented to the One-Stop Career Center Staff and the 
Central Texas Workforce Development Board. Examples of areas identified from the 
focus groups for Board consideration are as follows: 



1 Central Texas has been using customer satisfaction surveys and customer (and staff) focus 
groups to guide the strategic planning process since 1995. Plans are in the works to move “Beyond the 
Survey and Beyond the Focus Groups” and develop an “on-line” customer survey to augment their 
customer satisfaction measurement system. 
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Case Study Example: 



• The clearest challenge identified from the job seeker focus group 
discussion was the need to have personalized care and attention while 
still maintaining and/or developing the ability to serve large numbers of 
customers. 

• Job seekers identified difficulties associated with parking, accessibility, 
and service locations. 

• Employers desired access to enhanced services that are built upon those 
that are currently available. Elimination of current services was 
strongly discouraged. Primarily, employers were noted to want a 
system which provides rapid response for the identification of quality 
applicants 

• Employers described “quality” services as customer friendly, including 
access to computers and staff visiting employers on-site. 

• Employers recommended “team” service delivery - with staff members 
establishing habitual relationships with their employer customers; teams 
would include specialists to handle special needs. (Examples of 
specialist services include disabilities, veterans, technical assistance, 
etc.) 

• Front-line staff noted that the new system would require generalists 
(who know a little about all agencies and services) and specialists (who 
are experts in the requirements of a specific function and service). Staff 
felt that true integration would require a significant increase in cross- 
agency and cross-functional training as well as an increase in general 
information dissemination. 

Based upon the focus group responses and the front-line staff recommendations, 
a Customer Satisfaction Strategic Plan was developed that included specific 
recommendations to address challenges such as communication, accessibility, 
integration, training, job seeker and employer services. Copies of the focus group 
questions and the customer satisfaction survey results are provided on the following 
pages. 

In the words of a Central Texas employer, “in Central Texas we asked customers 
what they wanted, and began making what sells, as opposed to selling what we make.” 
(From Measuring Customer Satisfaction Beyond the Survey, Linda Angel, Central 
Texas Workforce Development Board, Belton, Texas) 

Three exhibits are included as examples of the tools used to collect customer 
feedback by the Central Texas Private Industry Council. Exhibit 3-1 lists the questions 
posed at focus groups with job seekers. Exhibit 3-2 lists the questions posed at focus 
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Case Study Example: 



groups with employers. Exhibit 3-3 lists the questions that were used in a mail survey 
of all JTPA participants and summarizes the customer feedback obtained from the 
survey. 

(Case Study Completed January 1999) 
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Exhibit 3-1 Module 3. 



Job Seeker Questions 

♦ Arc you aware of the services currently provided in the Central Texas One-Stop Career Centers? If 
yes, 

♦ What satisfied you most? 

♦ How could your experience with the One-Stop Centers have been better? 

♦ What more would you like to see One-Stop Centers provide in the future? 

♦ Has anyone had experience working with One-Stop Career Centers before? Why or why not? 

♦ What do you remember? 

♦ What went right? 

♦ What went wrong? 

♦ What kind of job seekers should use the Centers? Why? 

♦ Describe an ideal Center -the kind you would recommend to your best friend. 

♦ What about staff? 

♦ What about facilities? ^ 

♦ What about materials? 

♦ What about services? 

♦ Okay, now, what would “turn you off” about a One-Stop Career Center? Could you give an 
example? 

♦ What would be an acceptable time to: 

♦ Wait to be seen? 

♦ Receive information or services that meets your needs? 

♦ Expect results (for example, attain employment)? 

♦ Is there anything else? 
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Employer Questions: 

♦ Have you used any services currently provided in the Central Texas One-Stop Career Centers? If 
yes, what satisfied you most? 

♦ How could your experience with the One-Stop Centers have been better? 

♦ What would you like to see One-Stop Centers provide in the future? 

♦ Has anyone used a One-Stop Career Center like this before ? Why or why not? 

♦ What do you remember? 

♦ What went right? 

♦ What went wrong? 

♦ What kind of employers should use the Centers? WTiy? 

♦ What kind of job seekers should use the Centers? Why? 

♦ Describe an ideal Center -the kind you would use often. 

♦ What about staff? 

♦ What about facilities? 

♦ W'hat about materials? 

♦ What about services? 

♦ Okay, now, what would “turn you off” about a One-Stop Career Center? 

♦ Could you give an example? 

♦ How long should you have to wait for results? 

♦ Access to successful workers? 

♦ Receive i nformation or services that meets your needs? 

♦ How can the One-Stop Career Centers keep you involved in making it better? 

♦ Is there anythingelsc? 
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GRAND TOTAL BOTH OFFICES 

Number Surveyed: 181 Standard Deviation: 12.86 

Average: 84.67 (66% of all responses ranged between 71.81 and 97.53) 



AS = Agree Strongly; A » Agree; N = Neutral; D = Disagree; PS = Disagree Strongly 



QUESTION 


AS 


A 


N 


D 


DS 


1 . The JTPA staff treats me with respect. 


115 


59 


3 


4 




2. 


The JTPA staff acts in a professional and triendly manner. 


111 


64 


4 


2 




3. 


My needs were assessed and related services available to me through JTPA were discussed. 


90 


82 


7 


2 




4. 


1 was actively involved in determining the activities 1 would participate in to reach my goals. 


92 


78 


4 


5 


2 


5. 


My case manager provided career exploration or counselling about local job availability while 
helping me to develop my service strategy. 


63 


72 


34 


10 


2 


6. 


1 was given choices. 


76 


77 


15 


10 


3 


7. 


The JTPA staff listens to me about what 1 want. 


93 


74 


9 


3 


2 


8. 


1 was enrolled in the training 1 wanted. 


109 


59 


7 


4 


2 


9. 


1 feel the goals outlined in my service strategy are attainable. 


102 


73 


6 






10. 


1 feel the services 1 am receiving are appropriate with the goals identified in my service strategy. 


96 


77 


6 


2 




11. 


1 was told about services 1 could get from other agencies. 


47 


66 


34 


29 


5 


12. 


1 was encouraged to apply for financial aid to help me with my living expenses while in training. 


80 


75 


13 


9 


4 


13. 


My case manager cares what happens to me. 


100 


70 


5 


3 


3 


14. 


My case manager encourages my attempts to improve my life situation. 


97 


68 


13 


2 


1 


15. 


My case manager treats me in a manner which helps to improve my self-esteem. 


94 


71 


7 


5 


4 


16. 


1 feet the JTPA program is helping me to become independent and self-sufficient. 


108 


67 


6 






17. 


1 feel 1 can contact my case manager to discuss any problems which may stop me from reaching 
my goals. 


105 


63 


5 


4 


4 


18. 


1 am receiving support services through the JTPA program (if needed). 


79 


72 


19 


6 


5 


19. 


The training 1 am receiving is improving my chances for employment in the field 1 have chosen. 


123 


52 


5 


1 




20. 


1 feel 1 will succeed in meeting my goals with the continued support of the JTPA program. 


105 


61 


15 






21. 


1 find the JTPA program and staff helpful. 


101 


75 


5 






22. 


1 feel the JTPA program will help me compete effectively in the local job market. 


96 


78 


7 






23. 


1 believe that tax dollars are well spent on JTPA services. 


121 


52 


5 


2 


t 


24. 


1 feel my input about JTPA program services will be used to improve the local JTPA program. 


77 


89 


15 






25. 


1 cannot get the services 1 receive from JTPA through another agency. 


81 


65 


27 


7 


1 
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